This Page Is Inserted by IFW Operations 
and is not a part of the Official Record 

BEST AVAILABLE IMAGES 

Defective images within this document are accurate representations of 
the original documents submitted by the applicant. 

Defects in the images may include (but are not limited to): 

• BLACK BORDERS 

• TEXT CUT OFF AT TOP, BOTTOM OR SIDES 

• FADED TEXT 

• ILLEGIBLE TEXT 

• SKEWED/SLANTED IMAGES 

• COLORED PHOTOS 

• BLACK OR VERY BLACK AND WHITE DARK PHOTOS 

• GRAY SCALE DOCUMENTS 

IMAGES ARE BEST AVAILABLE COPY. 



As rescanning documents will not correct images, 
please do not report the images to the 
Image Problem Mailbox. 



Requested Patent: GB2320843A 
Title: 



ESTABLISHING REAL TIME VOICE AND/OR VIDEO LINK VIA THE INTERNET ; 



Abstracted Patent: GB2320843 ; 

Publication Date: 1998-07-01 ; 

Inventor(s): FALLER J; JONES ALAN L ; 

Applicants): IBM (US) ; 

Application Number: GB1 9960026923 19961224 ; 

Priority Number(s): GB1 9960026923 19961224 ; 

IPC Classification: H04M3/42 ; H04M11/00 ; 

Equivalents: 
ABSTRACT: 

The user of a Web browser often requires assistance by the representatives (or agents) of the 
company which owns the Web page. A real time voice and/or image link between the user's 
computer and an agent's computer is achieved by downloading an executable program code 
(eg. JAVA* small Greek phi *) onto the user's computer. When the user initiates the execution of 
the program code, the user's computer, under control of the program code, retrieves real time 
connection data from the user's computer and sends it to the agenfs computer. The agent can 
then initiate a real time voice or image link with the user using the real time connection data. This 
saves the user having to manually enter a contact name, address or telephone number. 



(12) 



UK Patent Application „„GB ,,,,2320843 „ 3 ,A 



(43) Date of A Publication 01.07.1998 



(21) Application No 9626323.8 

(22) Date of Filing 24.12.1996 



(71) Applicant(s) 

International Business Machines Corporation 

I Incorporated in USA • New York) 

Armonk, New York 10504, United States of America 

(72) Inventor(s) 

Alan L Jones 
J Falter 

(74) Agent and/or Address for Service 
S R Davies 

IBM United Kingdom Limited, Intellectual Property 
Dept, Hurstey Park, Winchester, Hampshire, 
S021 2JN. United Kingdom 



(51) INTCL 6 

H04M 3/4211/00 

(52) UK CL (Edition P> 

H4K KF42 

(56) Documents Cited 

EP 0740445 A2 WO 96/38018 A1 
"Filing HTML forms simultaneously", Jacobs et al. 
May 1996 INSPEC abstract no. C9608-6130G-009 
"Soothsayers for Oracle", Cetko, July 1996. DBMS 
vo», no.8> INSPEC abstract no. C9608-6160D-002 

(58) Field of Search 

UK CL (Edition O ) H4K KF42 KF422 KF50X KOB 
INT CL 6 K04L 29/06 , H04M 3/42 3/48 3/50 11/00 
11/06 11/06, H04O 3/00 
Online: WPI, INSPEC 



(54) Abstract Title 

Establishing real time voice and/or video link via the Internet 

(57) The user of a Web browser often requires assistance by the representatives (or agents) of the company 
which owns the Web page. A real time voice and/or image link between the users computer and an agent's 
computer is achieved by downloading an executable program code (eg. JAVA™) onto the user's computer. 
When the user initiates the execution of the program code, the user's computer, under control of the program 
code, retrieves real time connection data from the user's computer and sends it to the agent's computer. The 
agent can then initiate a real time voice or image link with the user using the real time connection data. This 
saves the user having to manually enter a contact name, address or telephone number. 

66 



28- 









30 






LAN 
















PBX 







VTA 

OrW File 



I Product A I 



fFttulfftdtrl 



\ 



\ 



1! 



A Co 1 * tieb top 



[Product A 



I Woe |[order\ | 



Qurry C 




O 

CD 

ISO 
CO 

ro 
o 

00 
CO 



> 



2/3 



User estabish es link with agent Lm? 

^ 



User receives Java applet [—104 




Applet searches for user data in system 
files and other user data files 



Applet prompts user fo 
if needed and confir 


r further information 
mation of user data 






Applet sends user da 


ta back to agent J, 



User is prompted to take modem 
off-line if necessary 



.116 



User receives incoming Internet phone call/ 
telephone call from appropriate agent call centre 



.118 



FIG. 2 



3/3 



Agent- server receives request for Web page~| ^_ 120 



Server sends out Web page and appropriate 
Java appl et to user 

[ Server receives query and user data |~024 



122 



Server notifies appropriate agent or agent I 
call centre of user query and user data 



126 



When requested by agent, server establishes 
real time communication link with user's computer 



FIG. 3 



2320843 



WEB INTEGRATED TELEPHONY 

This invention relates to web integrated telephony and is 
particularly relevant to business conducted over the Internet using the 
World Wide Web. 

Companies and organisations are increasingly advertising their 
products and services on the Web by publishing Web pages which are stored 
on Web servers. These pages are downloaded by prospective consumers who 
then browse the Web page using personal or network computers which run 
Web Browser programs. 

The user of a Web browser often requires assistance by a 
representative of the company which owns the Web page. This can be 
achieved by the user placing a telephone call to a number given in the 
Web page. The user may well then be faced with a series of selection 
instructions to enter numbers on the telephone keypad. These numbers 
generate DTMF tones which the company's Voice Response Unit (VRU) 
interprets and uses to find an appropriate available representative to 
assist the caller. During this period the telephone tariff is charged to 
the customer, or in the case of a 0500/0800 number, to the company. 
During peak periods the caller may be held in a queue for a substantial 
time before gaining contact with a suitable agent. 

Once the customer has made contact with an agent, additional time 
is spent in discussing the appropriate Web page, and section thereof, and 
time elapses while the representative locates and displays the relevant 
Web page, and investigates the customer's query. 

The current practice of publishing a telephone contact number also 
makes it more difficult or expensive to re-route the call to an agent 
located elsewhere, such as a consultant working at home. In some cases 
it is advantageous for a company to simply receive a telephone assistance 
request through the Internet, and locate an appropriate person, who 
properly briefed, can then return the call. 

Another way improves the user friendliness by semi -automatizing the 
process using embedded formatting commands such as hypertext mark up 
language. 

In this case, a user downloading a Web page using an Internet 
browser, the Web page showing a product or service on the screen. The 
user -clicks" on a control button or hypertext requesting a callback and 
a second screen is activated by the browser which has input spaces for 
name and phone number. The browser then sends the request when the user 
"clicks" on another control button. 
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After the user initiates a callback, the request is routed back to 
the Web page owners Internet server via e-mail /TCP- IP, which in turn 
sends the request to a Web unit that translates the Internet message and 
customer information into a predictive dial message (electronic copies of 
calls to make). An example of this is Rockwell Corp's Automatic Call 
Distributor. 

This goes some way to speeding up a query but still requires input 
from the user which requires time and effort and is subject to delay and 
error. Moreover if the user is making enquiries from more than one 
company or in some cases more than one product the information may have 
to be entered again. 

One solution is to store personal and business information in a 
recognisable and accessible format so that it may be sent to a business 
on request. This has been done in one form as an electronic business 
card as described by Versit on their Web pages but has not been 
incorporated into a communicating application. Full information 
instructing a computer how to make a real time connection with the user 
is not included in such a business card. For example, if the user only 
has one phone line and no Internet telephone he will need to disconnect 
when requested, if the user has an Internet voice phone or video phone 
the agent will need to know which type so that the correct protocol can 
be used. Other important information such as the Web page and the 
product detailing the query would be specific to that particular event 
and is not included in an electronic business card. 

According to the invention there is provided a method of 
establishing a real time voice and/or image link between a user's 
computer and an agent's computer comprising the steps of: the user 
establishing a computer network link between the user's computer and the 
agent's computer and downloading executable program code onto the user's 
computer; the user initiating execution of the program code on the 
user's computer; the user's computer, under the control of the program 
code, retrieving real time connection data from the user's computer and 
sending the real time connection data to the agent's computer; and the 
agent initiating a real time voice or image link with the user using the 
real time connection data. 

In this way the user minimizes the effort required to real time 
connection with an appropriate agent. The agent will have the advantage 
of previewing the query before contacting the user and so give a better 
response. Since the agent establishes the call, the user would not 
normally have to be burdened with the cost of the call. 
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Advantageously the real time connection data contains the telephone 
number of the user* By using a fixed link such as a telephone network a 
real time uninterrupted communication channel can be set up. 

The computer network is preferably the Internet so that 
geographical application of the method is potentially unlimited. 

The executable program code preferably instructs the computer to 
search for a particular data file. This particular data file would be in 
a known location and known format for ease of data retrieval. 

If the computer cannot locate a particular data file it will 
preferably search for real time connection data stored in alien data 
files. These alien data files would include data files associated with 
the operating system or well known applications that may require similar 
real time connection information for example, communication packages and 
Web browsers. Such an additional method makes connection set up much 
more user friendly. 

The executable program code is suitably a JAVA ™ applet. Web pages 
commonly contain embedded software or executable code known as 'applets' 
written in the JAVA ™ programming language. Such applets may interact 
with the user to collect information, and may communicate with other 
processes or applications on other computers attached to the network. 
Using a standard language such as JAVA ™ which is supported by a large 
number of Web browsers across many different platforms gives the widest 
possible user base for the invention. 

The agent may re-route the real time voice and/or image link from 
the user to a third party agent. This gives the agent flexibility with 
call allocation, for instance, if the agent's call centre is over loaded 
or a third party agent would better deal with the query. 

In order to promote a fuller understanding of the above and other 
aspects of the invention, an embodiment will now be described, by way of 
example only, with reference' to the accompanying drawings in which: 

Figure 1 is a schematic diagram of users connected via the internet 
and telephone network to a call centre network; 

Figure 2 is a flow diagram of a connection process from the user's 
perspect ive ; and 

Figure 3 is a flow diagram of the connection process from the 
agent's perspective; 
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Two main types of user are envisaged using the web integrated 
telephony (WIT) system (10) of this embodiment described (Fig.l). The 
first is a typical home user (12) with a home telephone (14) and home 
computer (16) both sharing a single telephone line (18) connected to a 
telephone network (20). The computer (16) is connected to the telephone 
line (18) and telephone network (20) via a modem (22) and a large 
computer network (24) such as the Internet via a service provider (26). 
The second type of user (28) is typically more sophisticated having a 
computer (3 0) connected to the Internet (24) over a dedicated line (32) 
through a local area network (34) and telephone (3 6) connected to the 
telephone network (20) through a private branch exchange (PBX) (38). 

An agent (40) is typically located at a call centre (40) with other 
agents each having a client personal computer or workstation (44) which 
is attached via a local area network to a server (46) . Typically the 
server (46) and client (44) use the well known TCP/IP protocol for their 
communications. Each agent (40) within the call centre (42) has a 
telephone (48) connected to a private branch exchange (PBX) (50) which is 
connected to the telephone network (20) to allow access to and from the 
external users (12, 28). The server (46) has a computer link to the PBX, 
possibly through an intermediate protocol conversion system (not shown) - 

A third party call centre (54) and a third party agent (56) are 
connected to the telephone network (20) and Internet (24) to provide an 
alternative call answering service if necessary. 

A well known and common example of the use of such an arrangement 
of a server (46) and PBX (50) (Fig. 1) is to provide information about a 
telephony caller on the terminal of the agent who is about to receive the 
telephony call. Thus when an incoming telephony call is received, it may 
be possible to identify the caller based on the calling or called number 
(ANI/DNIS information) . 

It will be appreciated that the server (46) and PBX (50) hardware 
illustrated in Figure 1 is very well-known in the art. One example is 
based on the Callpath system available from IBM, where for example the 
server (46) may comprise the Callpath Server/2 program running on an IBM 
PS/2 workstation or equivalent under the OS/2 operating system, or the 
AIX Callpath Server/6000 program running on a RISC System/6000 
workstation. Each client (46) may comprise an IBM PS/2 workstation or 
equivalent running the OS/2 operating system or the Microsoft Windows 
operating system, and a RISC System/ 6000 workstation running the AIX 
operating system. The clients and server may communicate using the 
TCP/IP protocol over a Token Ring or Ethernet local area network. 
Further information about the callpath products can be found in the 
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Callpath manuals available from IBM Corporation, such as IBM Callpath 
Coordinator for OS/2 and Windows General Information (order number GC22- 
0074) and the manuals referenced therein. 

As shown in Figure 1, the server (46) is connected to the Internet 
(24) . Such a connection is not required for operation as a conventional 
computer integrated telephony system, but is very straightforward for the 
skilled person to implement in view of the fact that the server already 
supports the TCP/IP protocol which is used for Internet communications. 
Such support is typically provided as part of the operating system 
running on the server, and is described for the AIX operating system (for 
example) in "AIX v4.1 System User's Guide: Communication and Network" 
(publication SC23-2545-01_ and B AIX v4.1 Communications Programming 
Concepts" (publication SC23-2610-01) (both available from IBM 
Corporation) . 

An example of the use of this embodiment of the invention is to 
enable Web initiated telephone assistance for a Web page (58) having an 
in-built Web Telephone Assistance (WTA) applet (60). When a browser 
accesses a Web page (58) which has a WTA applet (60), the applet is 
downloaded to the user's personal or network computer (10), and provides 
interaction with the user (12). 

The WTA applet (60) interacts with the computer (16) and user (12) 
to collect WTA data (62) such as: 

name, address of user, or company (62A); 
the user's account number (62B) ; 
the telephone number to call back (62C); 
brief description of query (62D) ; 

other information such as product and the Web page. 

Some of this data may be predefined by the user in a personalised 
WTA user data file (64) which is searched for on the user's file system. 
This aids the user in entering the minimum amount of information whenever 
a Web page with integrated telephone assistance is accessed. 

The WTA applet (60) then establishes a communication session with a 
Web Call Dispatcher {WCD) programme running on a server (46) connected to 
the Internet (24) and the TCP/IP address of this server (46) is coded 
into the WTA applet (60). Each Web page (58) which provides telephony 
assistance (24) can have one or more associated WCD servers. 

When a TCP/IP session has been established between the user's 
computer (16) and the server (46), the WTA data (62), including the data 
from the WTA data file (64), is sent to the WCD. Additional information 
in the WTA data (62) includes the Web page, Universal Resource Location 
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(URL) (58), details of the browser environment and necessary information 
entered by the user. 

The user's computer (16) under instruction from the WTA applet (60) 
now informs the user (12) that his/her call request has been registered 
and that a suitable agent (44) is being searched to deal with the 
request. If the tariff for the call is to be billed to the user (12) the 
user (12) may be asked to accept the charge. To accommodate for users 
who access the Internet (24) from a single telephone line (18) this 
message may also request the user hang-up the modem (22) connection to 
allow for an incoming voice call. 

The WCD programme uses the URL or the product information to locate 
a list of possible agents to service the request for assistance from a 
URL or product agent directory. The directory holds an updated list of 
agents, their Internet addresses, the associated telephone contact number 
for each Internet address, and a list of URL's or products which the 
agent has been trained to service, and a service category level for the 
URL or product (expert, etc) . The server, under instruction form the WCD 
programme, (in order of expert priority) establishes a TCP/IP session 
with a Web Page Agent (WPA) application (66) running on each agent's 
workstation. This application presents a pop-up window on the agent's 
display with details of the URL, the customer query (62D) and telephone 
number to call (62C) . The agent may then accept the request and deal 
with it manually (by manually calling the customer), accept the request 
with computer/network assisted connection, or decline the request. If 
the agent (44) cannot deal with the request the server tries the next 
agent on the list. If the agent (44) accepts the request for manual 
servicing, the WCS server (46) logs the request and ends processing of 
the query. 

In the case where the agent accepts the request with automatic 
connection, the following processing occurs: The agent's workstation 
(44) uses the URL it received from the server (46) to cause the agent's 
Web browser to access the relevant Web page. If this is successful, the 
WPA notifies the WCD server (46) that the agent can service the customer 
request. The WCD server (46) then uses the numbers of the agent and 
customer to establish a telephone connection between the two. This is 
achieved by the WCD server (46) communicating with a CTI (Computer 
Telephony Integration) server which controls a Private Branch Exchange 
(PBX) or Central-Office Exchange. The exchange switch- calls the agent's 
telephone number and when answered, calls the customer number. 

The agent (44) and user (12) are now in telephone contact and are 
viewing the same Web page (53) allowing the agent to provide help and 
guidance to the customer. 
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As an additional function, the WCD may support the ability for the 
agent to transfer the call, or conference-in, another agent. In this 
case, the agent's workstation sends a message for expert support to the 
WCD which searches for an additional URL service agent. If one is found, 
the initial agent may be treated in a similar way to the initial 
customer. The expert and agent, who now both have access to the relevant 
URL Web page, may decide to conference the call or transfer it. 

At the completion of the telephone call, the CTI server informs the 
WCD server which logs the call and ends processing. 

The method of the embodiment will be described in terms of the 
steps taken by the user's computer in establishing a real-time 
communication link with an agent with reference to Figure 2. The user's 
computer establishes a link with the server over the Internet (typically 
the World Wide Web) when a hypertext link and URL address for the server 
is selected, a web page typically showing a product range may then be 
downloaded (102) . To initiate a query the user selects a product from 
the Web page which starts execution of a WTA applet downloaded with the 
Web page (104). The user's computer, under instruction from the (WTA) 
applet, searches for WTA data (106) and if such data is found the user's 
computer prompts the user for confirmation and any other information if 
needed before sending the data to the server (102). 

If no such WTA data is found then the user's computer, under 
instruction from the WTA applet searches for the necessary drivers and 
software to make an internet phone connection (108) . The WTA applet will 
recognize more than one type of internet phone with their respective 
formats and protocols. If a recognized phone with the required user data 
is found then the user's computer prompts the user for confirmation and 
any other information (112) if needed before forwarding this data to the 
agent (114) . 

If no internet phone is located on the user's computer then the WTA 
applet instructs the user's computer to search files associated with 
other applications for the required user information (110). Such files 
include operating system files, modem files, auto-dial telephone files 
and communication package files. The user's computer prompts the user 
for confirmation of any user connection data found and any other 
information required (112) before sending this data to the server (114). 

Once an agent is ready to respond the user will receive an incoming 
internet phone call or telephony call according to the WTA user data 
(118). If the user has a single phone line and no internet phone then a 
prompt message will be received by the user's computer for the user to go 
off line (116) . 
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The method of the embodiment will also be described in terms of the 
steps taken by the agent's server in establishing a real-time 
communication link with the user with reference to Figure 3. Initially 
the agent's server receives a request for it's web page from the user 
(120) and sends out such a web page with appropriate applets to the user 
£122). The next stage typically involves the agent's server receiving, 
from the user's computer, a query on a particular product together with 
WTA data with details of how to set up a real time connection with the 
user (124). The agent server will select an appropriate agent to deal 
with the query and forward the query and user data to the agent's 
workstation (126). The server may select a third party agent call centre 
or a third part agent not within the local area network and forward the 
query and user data to the selected third party over the internet. When 
requested by the selected agent, the server establishes a real time 
communication link with the user's computer either via an internet phone 
or video or a telephony phone or video (128). 
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CLAIMS 

1- A method of establishing a real time voice and/or image link 
between a user's computer and an agent's computer comprising the steps 
of: 

the user establishing a computer network link between the user's 
computer and the agent's computer and downloading executable program code 
onto the user's computer ; 

the user initiating execution of the program code on the user's 
computer; 

the user's computer, under the control of the program code, 
retrieving real time connection data from a data file stored on the 
user's computer and sending the real time connection data to the agent's 
computer; and 

the agent initiating a real time voice or image link with the user 
using the real time connection data. 

2. A method as claimed in claim 1 whereby the real-time voice or image 
link is through a telephone network and the real time connection data 
contains the telephone number of the user. 

3 . A method as claimed in claims 1 or 2 whereby said computer network 
is the internet. 

4. A method as claimed in claims 1,2 or 3 whereby the executable code 
searches for a particular data file. 

5. A method as claimed in any one of claims 1 to 4 whereby the 
executable code searches for real time connection data stored on the 
user's computer in alien data files. * 

6. A method as claimed in any of the preceding claims whereby the 
executable code is a Java Applet. 

7 . a method as claimed in any of the preceding claims whereby the 
agent reroutes the real time voice and/or image link from the user to a 
third party agent. 



Tlie 

Patent 
Office 

fo 

Application No: GB 9626923.8 Examiner: Simon Rees 

Claims searched: 1-7 Date of search: 30 March 1997 



Patents Act 1977 

Search Report under Section 17 

Databases searched: 

UK Patent Office collections, including GB, EP, WO & US patent specifications, in: 
UK CI (Ed.O): H4K (KF42, KF50X, KF422, KOB, ) 

Int CI (Ed.6): H04M (3/42, 3/48, 3/50, 11/00, 11/06, 11/08), H04L (29/06), 
H04Q (3/00) 

Other: Online: WPI, INSPEC 



Documents considered to be relevant: 



Category 


Identity of document and relevant passage 


Relevant 
to claims 


A 


EP0740445A2 (ROCKWELL) Whole document, especially lines 
39-49 of column 2. 


1-7 


A 


WO96/38018A1 (ERICSSON) Whole document, especially from 
line 13 of page 4 to line 35 of page 5. 


1-7 


A 


INSPEC Abstract No.C9608-6130G-009, Jacobs et al, "Filing HTML 
forms simultaneously: Co Web-architecture and functionality", 
Computer Networks and ISDN Systems Conference, May 1996, vol.28, 
no.7-11, pages 1385-95 (see abstract). 


1-7 


A 


INSPEC Abstract No.C9608-6160D-002, Celko, "Soothsayers for 
Oracle [Java and Oracle tuning]", DBMS, vol.9, no.8, July 1996, pages 
20-26 (see abstract). 


6 




X Document indicating Jack of novelty or inventive step A Document indicating technological background andtor stale of the art. 

Y Document indicating lack of inventive step if combined P Document published on or after the declared priority date but before 

with one or more other documents of same category. the filing date of this invention. 

E Patent document published on or after, but with priority date earlier 

& Member of the same patent family than, the filing date of this application. 

An Executive Agency of the Department of Trade and Industry 



